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ABSTRACT 



This paper presents a status report on efforts to 



assess institutional effectiveness at the Community College of 
Pennsylvania (CCP) . Part I provides a description of the 
methodological and procedural considerations which have guided 
efforts to understand institutional effectiveness. First, a 
description is provided of the conceptual foundation for assessment, 
which emphasizes examining educational effectiveness from both the 
student and college perspectives, understanding students' educational 
goals, and determining cognitive and non-cognitive outcomes for 
students. Next, eight positive effects of institutional research 
efforts are described, including the development implementation 
of a set of all-college performance indicators, the design of new and 
continuing student goal statements, and the redesign of the program 
and discipline audit process. Part I concludes with challenges and 
future directions of CCP*s assessment efforts. Part II highlights 
some of the major institutional strengths and concerns identified m 
the first 3 years of assessment, including: (1) students reported 
greatest progress m developing communication skills and 
self-conf idenre; (2) CCP students entered the job market with 
relative ease and at salaries reflective of the economic value added 
by the college; (3) completion of a degree did not alv;ays mean 
improved performance at a transfer institution, and transfer 
performance varied across programs; and (4) all students reported 
dissatisfaction with the opportunities for integration into the 
campus life. Finally, part m provides a framework for future 
efforts m examining student academic progress at CCP, focusing on 
Tmto's model of student attrition. This model maintains that 
students who are more involved m a college's academic and social 
life are more likely to persist and to have higher levels of 
personal/academic growth than less involved students. The report 
concludes with an application of Tmto's theoretical framework to CCP 
students and a future research agenda. A 17-item bibliography is 
included . ( Author/JMC) 
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Preface 



During the past three years, the Office of Institutional 
Research in collaboration with a wide range of institutional 
staff has begun a comprehensive look at institutional 
effectiveness* This paper is a status report divided into three 



parts 



^^^^ ^ ~ A description of the methodology 

and procedural considerations 
which have guided efforts to date 
to understand institutional 
effectiveness* This part of the 
paper is taken from a 
presentation made by Thomas R. 
Hawk at the 1987 Annual Meeting 
of the Pennsylvania Community 
Colleges • 

P^^^ " Highlights someof thekey 

findings during the first three 
years of institutional 
effectiveness study* 

^^^^ " This section, prepared by Jane 

Grosset, describes a 
comprehensive model for examining 
student academic progress at CCP* 
It proviaes a foundation for 
future studies of institutional 
effectiveness* 



Office of 
Institutional 
Researc h 

September, 1^8^ 
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PAkT I 

A METHODOLOGY FOR EXAMINING INSTITUTIONAL EFFECTIVENESS 



INTRODUCTION^ 

The purpose of this section is to describe an institutional 
assessment program in action""what we have done to be^in to 
understand areas of institutional effectiveness and 
ineffectiveness. This report is very much about work in 
progress. After several false starts, we have started down a 
road which we believe will result in critical information 
essential to planning future institutional directions. We have 
already learned a great deal about institutional effectiveness. 
There is much more we need to know. 

This presentation covers three areas: 

A. How we decided that educational effectiveness should be 
measured at CCP including; 

- The conceptual model supporting our research; 

- Assumptions and organizational considerations which 
have significantly impacted on the approach being 
used. 

B. What we are specifically doing to assess quality and 
some indications of institutional impacts our findings 
are having. 

C. Challenges we have encountered and some future 
directions that we hope to take. 

The October, 1986 ETS Invitational Conference on "Assessing the 
Outcomes of Higher Education" did a remarkably comprehensive job 
of summarizing the leadership and research issues which must be 
resolved to do an effective j^b of assessing institutional 
performance. The Proceedings address several of the following 
issues in more detail. 



HOW WE DECIDED TO BEGIN TO ASSESS EDUCATIONAL EFFECTIVENESS 



One of the challenge in starting an effectiveness 
assessment initiative is the varying notiors which different 
constituents have about effectiveness. Examples of areas where 
effectiveness measures can be collected include: 

Financial Effe c tiveness (program cost faculty credit 
hour productivity, average clait. size, indirect 
cost containment, etc.) 
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Enrollment Effec tiveness (ability to achieve 



d n r" /-\ 1 1 ^ ^ ^ 



-argets, extent to which programs reach 
all appropriate populations in service area) 

Community Impact (economic impact on communitv, e.g., 
contribution to local GNP, extent to which' 
employer manpower needs are met, employer 
assessment of program quality) 

Educational Ef f ec t i ve ne s s- Tn .r , „^ , . , p „ , . ^ , „ , ^. „ _ 

^Are institutional expectations for exiting 

students in areas such as basic skills, general 
education and program specific competencies being 
achieved?) ^ 

Educational Effectiven ess - Student Per. nP.r.-„. - 
(Are students leaving the institution having 
achieved the educatio.al and personal goals they 
had when they enrolled at the college?) 



r oi, f^" ^''^u' ^^"'^ President, Judith Eaton, appointed an ad hoc 
task force charged to look comprehensively at issues of ^-^^ 
educational effectiveness- We were to begin a r es of studies 

r " grhs nd^r'r ^ '^^'^ -derstandfng of institutio a^ 

st.engths and weaknesses, and would point toward ways that 
institutional effectiveness could be enhanced. After a se-ies of 
i u i^" '^'"'"'^ discussions, we d e t e r m i ne d t h; t °' 
irinJipIes ^'^^^^^^-^^ -o.ld be guided by th. following 

Principle 1. A primary emphasis would be placed on educational 
erfectiveness examined from both the ctlTFg-e-^^d 
the students' perspective. 

Principle 2. A significant effort would be placed on 

understanding students' educational goals at -he 
t:me they first enrolled, and in understanding how 
.hey changed while they were at the College. 
Effectiveness would then be assessed in part based 
I'pon the relationship between students' 
educational goals and what they achieved. 

Principle 3. Educational effectiveness studies would focus on 
both cognitive and non-cognitive outcomes for 
students. 

- We felt that growth in the areas such as self- 
concept and the development of independent 
learning capabilities were as important to know 
about as cognitive development in generic skills 
and academic disciplines. 

Principle 4. Measures of educational effectiveness would be 
analyzed and reported by student s ub p o pu 1 a t i o n 
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with the expectation that, to be effective as an 
institution, we would not have to have the same 
performance for all studer.t groups^ 

- An easy example of this was the expectation that 
the percentage of student? successfully 
transferring from our Engineering Science 
program would be higher than that from our 
Office Admlnist ration program and vice versa for 
post-CCP employment. 

- A more difficult, and still somewhat unresolved, 
issue was associated with effectiveness measures 
for students who enter the College via one of 
our pre-College developmental programs. These 
students are more likely than others to have 
enrollment goals that are inconsistent with 
abilities. Thus, their goals may not always be 
appropriate as a standard for measuring 
institution's effectiveness. Further, there is 
not complete consensus within the institution 
about the College's expectations for remedial 
students. Stated somewhat simply--Is our 
primary goal to prepare students to take 
college-level courses--or is it to give these 
students the educational assistance they need 
(in whatever form) to ge^: on with their lives? 

In general, there are a wide range of student 
subpopulations with unique goals and educational 
needs. Assessment activities need to recognize 
the uniqueness of these different groups. 

Principle 5. Assessment efforts would proceed along several 
parallel, but complenentary, tracks. 

Track A. Creation of a series of across-the-board 

pe rformance i ndicators that would provide common 
information for all programs and disciplines at 
the Co 1 1 eg e . 

A key role for the performance indicators 
should be to serve as an alert ^nechanism to 
identify areas of the College where further 
assessment is warranted. 

Track B. Significant improvement to the periodic program 
and discipline evaluation process. 

One o^ the primary emphases has been on 
creating an evaluation process with 
substantial departmental faculty involvement 
in identifying, researching, and making 
recommendations on critical issues related 
to institutional effectiveness. 



Track In s t i u u t 1 o n a 1 Research studies to describe Che 

effectiveness of the College in a number of key 
areas* Examples of some research issues we are 
currently working on or have completed Include; 

Factors contributing to student retention 
and no n- retention; 

Contributions of the College to student 
growth in selected cognitive and non- 
cognitive areas; 

Relative effectiveness of various components 
of the College's educational programs and 
student services from a current and former 
student perspective; 

Student goal achievement patterns; 

Student transfer outcomes at Temple 
University. 

Track CI as s room- based research that focuses on tnc 

effect Iveness of the individual teacher. 

Patricia Cross, at the ETS Conference, 
argued forcefully for classroom- based 
assessment as an essential complement to 
institutional assessment v 



Understandably, there is not yet a broad 
institutional commitment to classroom- based 
assessment. Many faculty are concerned 
abo'it allowing researchers into their 
classroom because of Its potentially 
obtrusiv-e nature and/or because it may 
appear to be evaluative in nature. However, 
^t CCP, several smalJer-scale Initiatives 
have demonstrated that, under ideal 
condition!,, faculty members are willing to 
look critically at their instructional 
approaches and modify them based upon an 
honest assessment of student experiences in 
the classr oom * 



A critical characteristic of all classroom- 
based assessment activities has been that 
they were largely initiated by concerned 
faculty who, in turn, encouraged other 
faculty to participation. Institutional 
Research has been involved only to the 
extent of providing technical assistance 
upon request. 
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Principle 6. Our final principle related to methodology. We 

believed the methodology should hp manageable 2nd 
result in timely information* Two technical 
considerations have been important in our 
assessment e f forts ♦ 

First. We are giviwg significant weight to 
student judgments of educational 
effect iveness. As the ultimate 
bene f i c i ar ip s (or victims) of our 
programs, wt believe the students have 
critical information to give us about our 
effectiveness. 

Second. We believe that c r o s s - se c t i o n a 1 studies, 
where we look simultaneously at attitudes 
and outcomes of student cohorts at various 
stages in their education, can provide 
meaningful inrormat^'on. Longitudinal 
studies, where one group of students is 
stuuied over time, are important and 
sometimes more compelling in the richness 
of information they provide--but the 
timeliness and usefulness of information 
available from cross- sectional approaches 
make them important to our efforts. 

WHAT HAVE WE ACTUALLY DONE T O DATE AND SOME INSTITUTIONAL 
IMPACTS FROM O UR EFFORTS 

A • Developed and implemented a set of all- college 
performance indicators 

This activity has facilitated the program 
evaluation process by providing a common data 
base and has helped to identify critical trends 
at the discipline or program level that require 
special scrutiny and/or further analysis. 
Based upon institutional experience and 
reactions in using these indicators, they will 
be modified and/or expanded. 

^ • designed and imple m ented a new student ^oal 
statement as well a s introduced a conti nuing 
stu dent goal statement 

This has identified the types of enrollment 
goals which students are able to achieve and 
pointed to other areas where student progress 
has been less satisfactory. It has provided a 
firmer basis on which to assess student 
outcomes, e.g., studenus enrolling to take a 
few courses at the request of an employer 
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should not be expected to graduate. It has 
also emphasized the importance which CCP 
students attach to the o u t - o f - c 1 a s s r o om and 

J3 0Cia] d1mpnc:ionc: of rhoir" nH,,/^T<-n^r,^i 
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to deal with this issue* He 
Institutional Research has been able to confirm 
the validity of the overall student response 
patterns, if no: the reliability of individual 
cases . 



Undertaken a j^eneral education assessment for a 
cross-section 'of CCP students 

This effort demonstrated faculty willingness to 
participate in a general testing program and 
highlighted some areas of the College's general 
education experience which appears to be 
relatively weak (e.g., general knowledge in the 
Human 1 1 i e s ) ♦ 

Assessed student gains in the area of self- 
concept ■ - 

Consistent with other research, this study 
demonstrated the importance of placing students 
in a structured, curricular experience with 
close faculty ties, if student growth in non- 
cognitive areas is to be strongly encouraged. 

Redesigned the p?ogram and discipline audit 
pro cess 

The audits have become formative as opposed to 
summative in nature and are Intended to look 
critically at needed curricular and 
instructional changes, as opposed to being 
largely summative and se L f-app 1 aud i ng in 
nature . 



Constructed a oomprenensive Retention Data B ase 
w hich contains a wide range of demographic, 
entering characteristics, and academi c 
performance data on every stude nt e nro 1 1 1 ng a t 
the College on- and of f-campus during the lasT 
eight years^ 

With this data base, we can now unambiguously 
describe the academic progress of all students 
term by term, including off-campus, ccntinulng- 
education students for whom historically we 
have had limited information. 
6 

10 



- The data base makes it possible for us to load 
a rich set of variables for all respondents t: o 
survey research so that we look at the 
relationships between entering goals, race, 
sex, SES, remedial program status, etc., and 
educational outcomes. This is important to us 
because of the increasing emphasis we are 
placing on why something happened as opposed to 
just what happened. The data set is 
appropriate for causal mod ling techniques such 
as path analysis and Lisrel. 

^ ^ Initiated a current student survey and 

redesigned the former student sur veys "to gathe r 
much more detailed information on~students^ 
progress toward accomplishment of their goals 
for enrolling at the College, and to have a 
more detailed student assessment of various 
aspects of the College. 

For a range of student s u b po pu 1 a t i o ns , we have 
identified components of the institution that 
are relatively less effective. This, in turn, 
has helped to describe prioi^ities for 
strengthening curricula and support services, 

H • In collaboration with Tem p le University, 
undertaken a major'study oT transfer 
exper^-ences a't Temple of former CCP students. 

This study included data on 1,696 former CCP 
students who transferred to Temple during a six 
year period from Fall, 1979 to Fall, 1985, The 
study looked at the relationships between 
student background's, programs of study, and 
academic performance at CCP with subsequent 
transfer experiences at CCP. While the study 
findings are largely positive about the 
successful transfer experiences of CC^ 
students, a number of issues related to the 
adequacy of transfer education at CCP were 
raised. 
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CONTINUING CHALLENGES IN ASSESS INC INSTITUTIONAL £F F EC T I V E \^ £S S 



Russell Edgerton, President of AAHE, at the 1986 ETS 
Invitational Conference, stressed the long road that has to be 
followed to design and implement a meaningful institutional 
assessment model. As he said, "The quick fix just isn't in the 
cards." When CCP began its efforts to understand institutional 
effectiveness over three years ago, we: 

- underestimated the measurement complexity of some of Lhe 
effectiveness dimensions we wanted to explore; 

- underestimated the technical complexities associated 
with installing and accessing a large complex 
longitudinal data base; 

- overestimated the degree of sophistication at other 
colleges and universities that we thought we could turn 
to for help ; 

- overestimated the amount of interest our n o n - r e se a r c h- 
oriented colleagues would have in our efforts to improve 
our understanding of institutional effectiveness. 

Ve re lize now that we are involved in a long-term project where 
each set of research findings vi 1 1 raise new questions and point 
the way to additional research initiatives. 

There are several considerations which we believe are 
fundamental to the successful implementation of an institutional 
effectiveness assessment effort: 

1. Strong interest, support, and conceptual collaboration 
from the President have been critical in sustaining 
this assessment effort. Undertaking a comprehensive 
institutional assessment project has required a 
significant commitment of staff time and resources 
without any immediate tangible results. Patience and a 
tolerance for ambiguous information have been 
fundamental to getting through the first years of this 
institutional assessment program. 

2. An assessment process needs to balance the positives 
and the negatives. It is important that the assessment 
efforts be designed to look honestly at institutional 
deficiencies, but they should also provide a vehicle 
for highlight ing institutional strengths and 
accomplishments. Deciding on the balance between these 
two perspectives is an important ongoing consideration 
in developing an assessment model. 

3. To the extent possible, research methodologies should 
focus on attempting to establish causal relationships 
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as opposed to simply describing what happened* 
Possible approaches to strengthening curri rnU and 
services at the College should become apparent f.om the 
a s se s sme n t d a t a ♦ 

A- As the College develops and refines its expectations 
for students in higher order academic skills such as 
critical literacy, it is in>portant that the 
expectations be stated in such a way that they are 
realistic and measurable* All too frequently we are 
guilty of stating educational niceties as our goals 
with little or no attention paid to how these vague 
goals would De incjrporated into curricular design and 
assessment strategies. We need to avoid the "if we say 
it's important, we must be doing it" syndrome. 

5. A meaningful implementation of an institutional 

assessment process requires the support of staff and 
faculty if the results are to make a difference in 
institutional decision making. However, in designing 
institutional change strategies with resuect to quality 
assessment, many faculty and staff are core effective 
in a reacting mode than they are in a role of initially 
creating the process. In this first pnase, the Office^ 
of Institutional Research has played a leadership role 
in suggesting assessment strategies. A number of 
faculty and staff have been instrumental in helping to 
design current strategies. A continuing challenge is 
to identify appropriate junctures for expanded 
involvement in a way that allows good ideas to be 
captured and encourages ownership in the findings. 

With our work to date, we feel we have laid a foundation 
for a meaningful institutional effectiveness acjsessment program. 
What we are able to build upon this foundation, and what our 
ultimate institutional impact turns out to be will verv much 
depend on how well we resolve some of these latter issues. 



1) Publishea in 1987 by the Educational Testing Service. This 
document is entitled Assessing the Outc o mes of Higher 
Education . Copies should be requested from ETS , Princeton, 
N J , 0 8 5 4 1 . 
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PART II 



AN OUTLINE OF MAJOR CONCLDSIONS FROM THE COLLEGE'S 
EFFECTIVENESS STODIES 1985-1988 



Wichin Che concexcs of national research findings and 
student expressions of satisfaction, as well as tangible evidence 
of positive student outcomes, the College has been found to be a 
successful Institution in its service to a wide variety of 
student cohorts. The many studies that have been undertaken 
suggest that, with some very important qualifications, both the 
students and the College's educational goals for its students are 
being achieved. Nonetheless, within this broad sense of 
Institutional accomplishment, there are a number of findings both 
positive and negative which stand out and deserve to be commented 
on as key relationships developing from our institutional 
effective studies. 

Institutional Strengths 

1. Students who leave the College are generally satisfied 
with their overall experience. They give high marks to 
the College's instructional programs, their 
relationships with faculty, and to the economic 
mobility which che College has afforded the-. Many 
report their lives are fundamentally changed by their 
experiences at the College. 

2. At a uime when the transfer role of community colleges 
is seriously under question, CCP has maintained and 
supported this function well. Transfer rates exceed 
norms reported in any national studies. The great 
majority of students experience academic success when 
they transfer. The loss of credits is usually minimal 
for students who pursue studies at four-year colleges 
coni^istent with their program of study at CCP. 

3. Students who chose to enter the job market following 
CCP do so with relative ease and at entering salaries 
that are both acceptable and reflective of a 
significant economic value added by the College. 
Immediate employment outcomes following CCP are 
strongly Influenced by the students' choice of programs 
while at the College, with predictable differences 
among students enrolled in the various program areas, 
e.g.. Allied Health Program graduates earn more 
initially than do Human Services Program graduates. 

The great majority of students report a significant 
relationship between their study at CCP and their 
eventual job performance. (In interpreting employment 
outcomes over the last few years, it is important to 
remember that the Philadelphia area has enjoyed a 
period of remarkable economic robustness, creating 
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significant employment opportunities in some fields, 
where only a few years before, students were 
encountering more difficulty in finding employment,) 

^- Students have many and diverse expectations of how they 
will benefit from their CCP enrollment, including 
expectations related to personal and social 
development* Overall, students reported greatest 
progress in developing communication skills, meeting 
new and interesting people, developing se 1 f - co n f i a e nc e , 
and an openness to new ideas* Students, who continued 
enrollment at CCP into their sophomore year, were most 
likely to cite progress in improving their socio- 
economic status, developing clearer career goals, and 
increasing self-confidence. 

Many of the students enrolling at CCP with significant 
developmental education needs are eventually successful 
in completing Co 1 1 e ge - 1 e v e 1 studies. There is 
compelling evidence that the wide range of special 
academic programs and services offered to developmental 
students significantly impact on their retention and 
academic success at CCP. 

Institutional Concerns 

However, this general picture of institutional health and 
strength notwithstanding, there are a number of fundamental 
issues which emerge in the institutional effectiveness studies 
which suggest ways that the College could be strengthened. 

1. Despite the successful transfer of a high percentage of 
the College's former students, there are a number of 
transfer issues which need to be examined. Completion 
of CCP's degree does not always bring wfth it an 
expectation of improved performance at the transfer 
institution. Transfer performance varies across CCP 
programmatic clusters, with students from some 
programmatic areas experiencing significantly more 
difficulty in transferring than students do in other 
areas. This is especially true for students in rareer 
programs. Students in career programs appear to 
require academic program structures, guidance, and 
support services that are responsive to students with 
both career and transfer objectives. When 
transferring, they experience a greater loss of credit 
and have a reduced probability of graduating with a 
bachelor's degree. The inconsistent loss of credits by 
students, even those in the same programs, suggests 
that transfer advising is not uniformly occurring 
across the College. This is evidenced by the lack of 
satisfacuion expressed by some categories of students 
with transfer advising. A significant percentage of 
students encountered difficulty with their 
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communications skills 

both their own reported sense of lack of progress 
developing writing and oral skills and their 
performance on writing placement tes^.s at Temple 
University* 

Across almost all measures of student performance 
minority students, especially Blacks and 
Hispanics, have less success. This is reflected 
In their placement in dropped for poor progress 
and academic standard status, graduation rates, 
completion rates of critical general education 
courses, s e 1 f - r e p o r t e d progress on a number of 
critical institutional goals, withdrawal rates 
from courses, and post-CCP employment experiences 
Minority students with an Asian background 
displayed very different outcome patterns. Their 
major academic deficiency Wcjs in the area of 
written communication skills. They were, by far, 
the most focused student subgroup with regard to 
goals and intentions and we re tlie group most 
likely to plan to transfer. 

In many respects, sophomore students are less 
satisfied with their experience at CCP than 
freshmen. This is reflected in their evaluation 
of the College's programs and services; their 
reported lack of progress on a number of key 
institutional goals such as ability to critically 
evaluate ideas; and their frustration with 
opportunities for o u t -o f - c 1 a s s r oora experiences at 
CCP and, ultimately, may be implicated in their 
lack of retention at the College. 

To date in institutional research findings, there 
has been little evidence developed that supports 
clear student advantages to earning a CCP degree 
beyond those achieved by students who enrolled at 
CCP for a significant number of credits but elect 
not to graduate. Few differences have been found 
for graduates and nongraduates (who earn a 
significant number of credits at CCP) in transfer 
outcomes including transfer grade point averages 
or graduation from the baccalaureate institution. 
With respect to career outcomes, reported salaries 
and the relationship of jobs to CCP program of 
study are similar for both groups. (There are 
some career program areas--most notably Allied 
Heal th--wher e these comments do not apply.) There 
Is, of course, no guarantee that non-graduating 
students have fulfilled the College's general 
education expectations. 
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5. All students report dissatisfaction with the 
opportunities for social integration with the 
institution. They expressed a desire for more 
involvement in activities as well as for informal 
gathering places to interact with other students. 
Even students who are in sma 1 1- c lo i s t e r e d 
programmatic environments, where academic success 
at CCP as well as post-CCP outcomes in many 
respects tends to be enhanced, report 
dissatisfaction with the lack of social 
experiences at the Co liege. 

6. While the structured, close-knit programmatic 
experiences offered in some areas at CCP promote 
retention and academic success at CCP, this 
programmatic experience is not always associated 
with acadejnic success upon transfer, nor 
associated with overall student satisfaction. It 
is possible that the uni-dimensional experiences 
in some of the t i g h t 1 y- co n f i n e d curricular 
experiences may not be addressing all aspects of 
student aevelopment that are conducive to 
maximizing their satisfaction while at CCP or 
their ultimate transfer success. The College may 
not be adequately preparing some students for the 
required level of independence and se 1 f -d i r e c t i o n 
required at large, four-year colleges. 

7. There is significant variability across race in 
both students* reasons for enrolling at CCP and 
expectations for their college experience. These 
findings, coupled with the tremendous vario»nces in 
performance, support the potential gains that may 
result from retention and academic success 
initiatives focused ipon specific minority groups. 

8. Student usage of support services are often not 
consistent vith student goal expectations. More 
of an effort is required to encourage students to 
utilize appropriate institutional supports that 
can assist in the achievement of their enroll me nt 
goals . 

9. The College's remedial and developmental programs 
are playing a crucial role in preparing a 
significant number of students who are eventually 
able to go on and complete college-level studies. 
However, for those students who do not 
successfully complete a significant percentage of 
college-level courses, there is currently a lack 
of evidence which suggests that they have been 
well-served by the current programmatic offerings. 
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10, A number of recent studies suggest that :2any 
students wouid benefit and appreciate a 
strengthening of the general education experience 
offered at the College. Many students report a 
lack of progress, especially in the sophomore 
year, toward the accomplishment of general 
education goals such as the improvement of 
communication skills. Furthermore, standardized 
testing results as well as an examination of 
student success in general education courses 
indicated deficiencies in this area. A sample of 
CCP sophomores tested below sophomores at other 
community colleges in one assessment activity 
utilizing a national test program. Humanities was 
the weakest area of student development, followed 
by the areas of science, social sciences, literacy 
and math. Possible student weakness in basic 
skills was suggested by an analysis of former 
student performances on Temple'j placement tests. 

11. CCP appears to be doing a better job in preparing 
students for transfer than in the area of career 
readiness. While large numbers of students find 
employment after leaving CCP, a small, but 
significant, percentage a- 2 unemployed but 
actively seeking a job, or are employed outside 
their field of study. Not surprisingly, students 
from the transfer and the General Studies 
curricula are less likely to find education- 
related jobs and generally earn lower salaries 
than students in career programs. Just as we need 
to intercede with career students who plan to 
transfer, we may need to provide better support to 
transfer students who decide to work following 
CCP. 

We continue to know more about on-ca.npus students than 
we know about those enroll off-campus. Students who enroll 
in the Community Services Division generally continue taking 
Community Services courses and rarely enroll on campus. 
Outcomes associated with these students are generally quite 
distinct from students enrolled on campus. They receive 
better grades and are less likely to withdraw from courses. 
Transfer outcomes, such as graie point averages and 
graduation from the four-year school, are generally superior 
to Main Campus students as is their academic performance 
while at CCP. However, in terms of multi-term retention and 
persistence, they remain at the College for a much shorter 
period of time. This appears to reflect both different 
student goals and lack of programmatic opportunity at off- 
campus locations. 
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A majority of CCP outcome <?t-!j<jies have been focused 
to date, on students completing 12 credits o r " mo r e " Thi s ' 
approach has been based upon the assumption that, 
significant impacts could not reasonably be attributed to 
CCP for students who completed fewer credits at CCP. 
Unfortunately, this means we currently know little about 
those students who left CCP computing no or only a few 
credits. Because this group is large and retention is an 
important institutional c o nc e r n - - th i s is an important next 
phase for institutional effectiveness studies. 
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A CONCEPTUAL FRAMEWORK TO GUIDE FUTURE 
EFFECTIVENESS STUDIES 

INTRO DU CT ION 

It was always our intention to provide more than a 
documentation of success measures which characterize the 
current state of institutional effectiveness at CCP. 
Anticipating from the start that the data would suggest 
areas of relative Ineffectiveness in institutional 
functioning, and that College faculty and staff would be 
genuinely interested in providing rnorc effective growth 
opportunities for students, it became Important to recast 
the descriptive information that was assembled into a 
policy-oriented framework that would provide meaningful and 
workable suggestions for Improving institutional 
effect iveness . 

Shotland and Mark (1985) suggest that the creation of 
useful social programs and policies is enhanced bv the use 
of theoretical frameworks that focus on process variables 
that policymakers are able to manipulate. For our 
assessment, it would therefore be Important to focus on 
institutionally controllable influences on student growth. 
I.e., collegiate experiences over which the College has some 
policy or programmatic control and which might facilitate or 
Impede the attainment of desired educational goals. 

Tinto's model of student attrition provides a useful set of 
guiding principles foi conceptualizing potentially influential 
va.-iables and processes in student growth. Numerous studies 
exist in the retention literature, many of which represent 
exemplary research projects (Bean, 1 98 2, 1 983, 1 985; PascareUa 
and Chapman, 1983; Pascarella and Terenzini, 1980 1983- 
Terenzini and Pascarella, 1977, 1978) that generally validated 
all or part of ; >e model. Although the model, as designed by 
Tiptv., explains undergraduate student attrition, several 
researchers (Terenzini and Wright, 1 987; Theo phi 1 id e s , et al . , 
1984) have adapted it for studying institutional impact~n~~ 
student gains in cognitive and n o n- cog n i t i v e areas of development 
as well. This broad-based empirical support for the model is 
important given the diverse dimensions of effectiveness (academic 
progress, retention, transfer, and career outcomes) that are 
being measured at CCP to assess Institutional effectiveness. 

The model is based on the premise that if the college 
experience positively influences a student's academic and 
personal growth (as most educators purport), it is reasonable to 
expect that the student who is more involved in the academic and 
social life of the college will both be retained with greater 
probability and experience per^^onal growth more than a less 
Involved student. 



conceptual framework of the model is complex and a 
detailed discussion of it follows. Gi-en ics roots, th^s 
presentation centers on the model's application in the area of 
student retention; however, the reader is reminded that the other 
effectiveness measures can be substituted as the outcome 
varl able . 



Tinto ' s Mod el of St ud ent Attrition 

Tinto views the dynamics of attrition from an 
interact ioni St • s perspective. Colleges are like other human 
communities and student departure is a reflection of both the 
attributes and actions of the individual and those of the other 
members of the community with whom the individual interacts. As 
such, Tinto believes that retention rates serve as an indicator 
of institutional quality and health, and that retention studies 
reveal much about the character and problems within a college. 

The model purports that two factors are the primary causes 
of individual withdrawal from college: personal attributes of 
students which predispose them to respond to given situations or 
conditions with particular forms of behaviors, and the nature of 
student interactional experiences within the institution 
following entry. 

Two categories of personal attributes which predispose some 
students toward departure are intentions and commitments. 
Intentions are aspirations, most often stated in terms of 
educational and occupational goals, toward which individual 
activities are directed. Commitments represent important aspects 
of personality which incline a person toward completion of tasks 
once begun. It i e important to consider student commitment to 
both their educational goal(s) and their commitment to a specific 
institution. 

Students enter college with intentions and commitments 
which set the limits of their educational attainment and 
influence their experiences within the college. Intentions and 
commitments, however, are subject to change. Over time, they 
reflect the character of student experiences within the 
Institution. Though prior student intentions and coram i trnent s raav 
lead directly to departure from college, Tinto assumes their 
principle impact is contingent on the quality of student 
interactions with other members of the institution following 
entry and on the students' assessment of the degree to which 
those interactional experiences meet his or her needs and 
interests . 

This concept of s t ud e n t - i n s t i t u t i o n al fit is central to the 
model. Tinto believes that integration is an important 
interactional outcome that arises from an individual's 
experiences within the institution. Integration is the degree to 
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which intellectual and social experiences within the institution 
asflimTlate students into the intellectual and social life of the 
inst i t ut io'i . Theorstlcally, for two students with similar 
Intentions or goals and the same levels of initial commitments, a 
higher degree of integration into the college systems for one 
would mean greater subsequent educational goal and institutional 
commitment . 

Absence of integration arises from two sources: 
Incongruence and isolation. Incongruence results from a mismatch 
between the abilities, skills and interests of the student, and 
the demands and opportunities of the academic and social systems 
of the college. Isolation occurs where there is an absence of 
sufficient contact between the student and other members of the 
social and academic communities of the college. Tinto views some 
level of incongruence-related institutional departure as 
inevitable. On the other hand, he feels i s o 1 a 1 1 o n- r e 1 a t e d 
departure need not occur. 

Although the model assumes that the decision to withdraw is 
largely the result of events which take place within the 
institution following the student's entry, Tinto believes it also 
reflects the pre-entry attributes and skills of students and the 
pressure of external student commitments. The model therefore 
Includes several categories of variables: pre-entry attributes, 
initial goals and commitments, academic integration, social 
integration, and later goals and commitments. Tinto causally 
links these variable categories in a longitudinal fashion. The 
hypothesized flow of events, and their direct and indirect impacts 
are indicated by the arrows that appear in Figure 1. 

Student pre-entry characteristics include measures related 
to family and community background. Intellectual and social 
skills, and varying types of pre-college educational experiences 
and achievements. These background characteristics interact witb 
each other, as indicated by the directional flow implied by the 
arrows in Figure 1, and influence the ways in which students 
interact with the college environment by directly effecting 
initial intentions and commitments. 

Student intentions and commitments, in turn, directly 
effect the subsequent Interactions between the student and other 
members of the institution. It is through these interactions 
with the environment that students become Integrated to varying 
degrees into the system both academically and socially. 

The academic system of the college centers around the 
activities that are concerned with the formal education of 
students. It Involves faculty and staff who are primarily 
responsible for the training of students. 

The social system centers about the daily life and personal 
needs of the various members of the institution. It includes 
Interactions among students, faculty and staff, which take place 
outside of the academic domain of the college. 
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The model distinguishes between the formal and Informal 
aspects of each system. The formal social system of the coJlege 
represents structured, extracurricular activities, while thp 
Informal system includes the day-to-day casual activities among 
the members of Che colloge. Classrooms and laboratories set ths 
boundaries of the formal academic domain of the Inotltution, 
while informal academic interactions occur outside the formal 
instructional setting. 

It is assumed that these student Interactions in the 
academic and social systems of the college continually act f.o 
modify their goal and institutional commitments in ways which 
lead to persistence or dropping out. The experiences of students 
in each system may have quite separate effects upon their 
persistence or withdrawal from the Institution. For example, in 
the academic system, inlt ^1 goal co- itment leads to higher 
grade performance and intellectual development; which leads, in 
turn, to academic integration; which, in a circular fashion, 
leads to even greater goal commitment. In turn, goal commitment 
reduces the likelihood of dropping out. In the social system, 
institutional commitment is expected to produce peer group and 
faculty Interaction; which leads to social integration; which, in 
turn, increases institutional commitment. This is also expected 
to reduce the likelihood of dropping out. 

Since intentions and commitments are not assumed to be 
fixed, they appear twice in the model. Their first appearance is 
the product of pre-entry characteristics, while the second 
appearance of these variables is assumed to be the product of 
academic and social experiences. 

The model does not argue chat full integration in both 
systems of the college is necessary for persistence. Nor does it 
claim that failure to be integrated in either system necessarilv 
leads to deparCure. RaCher it purports chaC some degree of 
social and incelleccual incagracion must exlsc as a condicion for 
concinufcd persistence. A rauCually compensatory relacionship is 
hypochesized beCween social and academic inCegraClon in chaC a 
low level of academic inCegracion wichin a given insCicuCion may 
be compensaced for by a corresponding high level of social 
inCegraCion and vice versa. 

Though Che model emphasizes che role of 1 n C r a- i n s C 1 c u C i o na 1 
experiences, ic does noC exclude che possibility chaC exCernal 
evencs can also influence individual decisions regarding 
deparcure. Social forces exCernal Co che insClcuCion may 
Influence sCudenC decisions regarding behav'ior in che 
Insc icucional seCClng. ExCernal communities including families, 
work seCClngs, and peer groups may serve Co counCer, rather Chan 
supporc , pare lei pac ion in college communicles. This is 
especially so when che requiremencs of membership in ar. exCernal 
communlcy are counCer Co chose for membership in an i n s t i C u t i on a 1 
community. A form of role conflict may develop and the student 
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may be faced w5th having to choose between college participation 
and participation in non-college activi^ies. The strain of such 
conflicts may undermine integration in college. 

The model assumes that the impact of these external 
commitments do not directly affect the decision to drop or 
persist. Instead, their influence on drop decisions is observed 
in the students changing intentions and commitments. 

Appropri a tene ss o f t he Mode 1 for CCP 

Many of the effectiveness results, when viewed from a 
correlational perspective, exhibit patterns which are consistent 
with the conceptual framework of the model. Much of the data 
appears to support the importance of the Lwo-core concepts of 
academic and social integration. Consider the following set of 
independently-collected facts concerning CCP freshmen and 
sophomore student differences. 

- Sophomore students reported in surveys that they were 
less satisfied with their overall CCP experiences, 
academic and social, than freshmen. 

- Sophomore students in the same survey reported a lack of 
progress on a number of important institutional goals 
such as the ability to critically evaluate ideas. 

- Less than half of entering students persist at CCP to 
achieve sophomore status (24+ cumulative registered 
hours) . 

The following set of information concexning differences in 
remedial student outcomes also appears to lend support to the 
importance of student academic and social support. 

CCP students who entered at a remedial level and 
participated in the College's remedial and 
developmental programs persevered at CCP for the same 
number of credits as students who entered at a 
college-re ad ylevel. 

Remedial-level students enrolled in support programs 
achieved greater levels of academic success than 
remedial-level students who did not receive support 
through CCP developmental programs* 

Students at a remedial level who did not have the 
support of developmental programs dropped out sooner 
than remedial-level students with support. 

The descriptive/correlational data that have been assembled 
during the past several years represent one of the first steps in 
the evaluation of institutional effectiveness at CCP. This 
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effort has provided a valuable framework for future research 
efforts by giving insight into both the differential 
effectiveness across student subgroups and important 
institutional characteristics that n.ay contribute to student 
development processes* The next steps in the evaluation of 
institutional effectiveness are studies chat begin to examin 
these seemingly important factors for their degree of causal 
influence on student development* 
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Fut ur e Research Agenda 

In response to institutional information needs related to 
the 1988-91 Strategic Plan, several potential student subgroups 
have been identified for study using Tinto's theoretical 
framework . 

The issue of how the College should respond to matriculants 
who are not interested in earning a degree has been targeted as a 
Strategic Plan initiative. A subset of this student group 
includes those who leave CCP completing fewer than 12 credits. 
While prior institutional research has indicated there have been 
a sizeable number of former students who meet this criterion, 
many outcome studies have not included this student group. 
Consequently, much of what we know about former students is not 
representative of short-term enrollees, thereby creating a 
considerable information gap. 

The differential success rates associated with minority and 
majority CCP students across a broad range of outcome areas has 
been well documented in institutional re tarch over the last 
several years. Consequently, an initialive set forth in the 
Strategic Plan at CCP addresses the issue of how institutional 
efforts might be strengthened in order to encourage the 
enrollment and academic success of minority students. In order 
to approach this issue with a coherent plan, it would be helpful 
to better understand the nature and causes of these differences 
in order to develop effective reforms. 

Another question raised within the framework of the 
Strategic Plan is how CCP should respond to the growth in numbers 
of potential CCP students having s pe c i a 1 - ed uc a t i o na 1 needs. 
Prior institutional research has indicated that a considerable 
segment of r sm ed i a 1 - 1 e v e 1 students have benefitted from 
enrollment in the structured services provided by Educational 
Support Service (ESS) prograrrs. However, not all r e m ed i a 1 - 1 e ve 1 
students are served successfully or at all by the ESS programs 
and the outcomes associated with many of these latter studt^nts 
fall well short of the successful end of the outcome continuum^ 

Future institutional research efforts will include 
developing and testing causal models of interrelationships among 
the categories of variables suggested as important by Tinto 
(student background, goals and commitments, academic integration, 
and social integration) to the process of student growth. These 
results will, hopefully contribute to the development of 
institutional strategies designed to enhance institutional 
effectiveness for these various student s u b po p u 1 a t i o n s . 
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